Skip to content

Custom events, PostHog exporter, async identify, and hardened event pipeline#37

Open
naji247 wants to merge 3 commits into
mainfrom
feat/custom-events-posthog-reliability
Open

Custom events, PostHog exporter, async identify, and hardened event pipeline#37
naji247 wants to merge 3 commits into
mainfrom
feat/custom-events-posthog-reliability

Conversation

@naji247

@naji247 naji247 commented Jul 8, 2026

Copy link
Copy Markdown
Member

Summary

This PR brings the Python SDK to feature parity with the TypeScript SDK and hardens the event pipeline so that no SDK failure can ever surface into a customer's MCP server. It adds custom event publishing, a PostHog exporter, async identify support with per-request identify events and identity merging, fixes a bug where redaction functions never ran on published events, and adds an extensive failure-injection test suite.

New features

publish_custom_event

Publish agentcat:custom events for actions outside the MCP request lifecycle:

from agentcat import publish_custom_event, CustomEventData

publish_custom_event(server, "proj_...", CustomEventData(
    resource_name="nightly_sync",
    message="Scheduled sync completed",
    duration=1240,
    tags={"pipeline": "nightly"},
    properties={"records": 1832},
))

The first argument accepts either a tracked server or an MCP session ID string. String session IDs are converted with the same deterministic SHA-256 → KSUID derivation used by the TypeScript and Go SDKs, so custom events correlate with tracked sessions across languages and server restarts.

PostHog exporter

Joins the existing OTLP, Datadog, and Sentry exporters:

options = AgentCatOptions(exporters={
    "posthog": {"type": "posthog", "api_key": "phc_...", "enable_ai_tracing": True},
})

Sends capture events with mapped names (mcp_tool_call, mcp_initialize, ...), person profile updates from identified actors, $exception events on errors, and optional $ai_span AI-observability events. Session IDs are mapped to deterministic UUIDv7 values.

Async identify + identity merging

  • identify hooks may now be async def; sync hooks work unchanged.
  • The hook runs on every captured request, and an agentcat:identify event is published each time it returns an identity. Identities merge across calls — user_id/user_name are overwritten by the newest result while user_data keys accumulate — with an LRU cache (1000 sessions) providing merge continuity across server-object recreation. Actor attribution on individual events is unaffected.
  • In stateless mode the cache is bypassed and each request's raw identity is published as-is, so user_data from different actors sharing a server instance can never merge together.

Bug fixes

  • Redaction now runs on published events. The redaction pass only traversed plain dict/list/str structures and silently skipped the event model, so redact_sensitive_information functions never ran on the live publish path. The three e2e tests documenting this were previously marked expected-failure and now pass. Async redaction functions are also supported and awaited.
  • Full structured errors on the low-level Server path. The low-level integration recorded only the error message; it now captures the same structured data (stack frames, chained exceptions, in-app detection) as the FastMCP integrations.
  • publish_custom_event argument errors are confined to the documented ValueError/TypeError contract.

Truncation alignment

Event truncation now applies dedicated per-field limits before the total-size pass — user intent and error messages (2048 chars), resource/server/client names (256), stack frames (50, keeping first and last 25), response text blocks (32KB) — and the general limits were aligned with the published event contract: 32KB per string, depth 10, breadth 100, 100KB total (previously 10KB / 5 / 500).

Reliability hardening

The SDK's contract is that analytics must never break the host server. This PR closes every gap found in an adversarial review of the request path:

  • Exception capture, identify hooks, tag/property callbacks, and event publishing are all fully fail-open: failures are logged and the event is dropped, and the customer's tool call proceeds untouched — including pathological cases like exceptions whose own __str__ raises, callbacks returning non-dict values, or objects with a raising __bool__.
  • The queue worker survives poisoned events (cyclic structures, NaN/Inf, bytes, objects with raising __repr__, redaction functions that raise sync or async) and keeps delivering subsequent events.
  • Exporter failures are isolated per exporter and never affect delivery to the API or other exporters.

Each fix is covered by tests/test_failure_injection.py (38 tests), several of which exercise real Streamable HTTP transports end to end.

Compatibility notes

  • An agentcat:identify event is published for every captured request where the identify hook returns an identity. Per-event actor fields are unchanged, and identify events now carry the accumulated (merged) user_data.
  • Users who configured redact_sensitive_information will now see redaction actually applied to published events.
  • Event payloads may retain more detail due to the raised truncation limits; the 100KB total event cap is unchanged.
  • No breaking API changes. New exports: publish_custom_event, CustomEventData, PostHogExporterConfig.

Testing

  • Full suite: 611 passed, 4 skipped (skips pre-existing/environmental), including e2e suites over stdio and Streamable HTTP for the official FastMCP, community FastMCP v2/v3, and low-level Server integrations.
  • New suites: failure injection (38), custom events (16), PostHog exporter (22), identify semantics and merging (14+), low-level error capture, field-level truncation (19).

naji247 added 3 commits July 8, 2026 16:42
…vent pipeline

Features:
- publish_custom_event(): publish agentcat:custom events against a tracked
  server or an MCP session ID string, with support for message/user intent,
  parameters, response, duration, error state, tags, and properties. Session
  IDs derived from MCP session IDs are deterministic (SHA-256 -> KSUID), so
  custom events correlate with SDK-tracked sessions across languages.
- PostHog telemetry exporter: /batch capture events, person profile updates,
  $exception events on errors, and optional AI-observability spans behind
  enable_ai_tracing. Session IDs mapped to deterministic UUIDv7.
- identify hooks may now be async; sync hooks keep working unchanged.
- Identity change detection: identify events are published only when the
  resolved identity actually changes; user_data is merged across calls and
  identities are cached (LRU, 1000 entries).

Fixes:
- redact_sensitive_information is now applied to live events. Previously the
  redaction pass silently skipped the event model, so custom redaction
  functions never ran on published events. Async redaction functions are
  now supported and awaited.
- The low-level Server integration now captures full structured error data
  (stack frames, chained exceptions) instead of only the error message.
- Event truncation applies per-field limits (user intent, error message,
  resource and server/client names, stack frames, response text blocks) and
  raises general limits to match the published event size contract
  (32KB per string, depth 10, breadth 100, 100KB total).

Reliability:
- The request path is fully fail-open: failures in identify hooks, tag and
  property callbacks, error capture, or event publishing are logged and
  dropped without affecting the customer's tool call - including exceptions
  whose own __str__ raises.
- The queue worker survives poisoned events (cyclic structures, NaN/Inf,
  bytes, objects with raising __repr__) and continues delivering
  subsequent events.
- publish_custom_event raises only its documented ValueError/TypeError
  contract on invalid arguments.

Tests: 611 passed, 4 skipped, including new failure-injection, custom
event, PostHog, identify, and truncation suites.
The identify hook already runs on every captured request; an
agentcat:identify event is now published each time it returns a valid
identity, instead of only when the identity changed. Merge semantics
are unchanged: user_id/user_name are overwritten and user_data keys
accumulate across calls, with the LRU cache retained for merge
continuity across server-object recreation.

Stateless mode now bypasses the identity cache and publishes the raw
identity per request, so user_data from different actors sharing a
server instance can no longer merge together.

Tests updated to the new semantics, including a stateless
no-cross-request-merge case. Full suite: 611 passed, 4 skipped.
Code-quality pass over the branch; no behavior changes.

- exceptions reuses logging.safe_error_string instead of a local copy.
- Async redaction functions now cost one event loop per event instead
  of one per redacted string; sync paths are unchanged.
- Redaction rebuilds the event with model_construct, skipping
  re-validation of data that came from an already-valid model
  (serialization verified identical).
- PostHog exporter computes the session UUID once per export.
- publish_custom_event normalizes its optional payload once instead of
  guarding every field access.
- Event-type validator generalizes to the set of SDK-defined event
  types instead of special-casing one constant.
- Removed dead IdentityCache.__contains__ and a redundant second parse
  of error results on the low-level path.

Full suite: 611 passed, 4 skipped.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant